Systems and Methods for Friction Displays and Additional Haptic Effects

ABSTRACT

Systems, methods, and computer program products for providing composite haptic effects are disclosed. One disclosed method includes detecting a touch occurring in a touch area when an object contacts a touch surface and selecting a composite haptic effect to generate in response to the touch, the composite haptic effect including at least one surface-based haptic effect and at least one other effect. Based on the selected composite haptic effect, a first haptic signal can be sent to cause an actuator to vary a coefficient of friction of the touch surface and a second actuator can be caused to provide a second haptic output in addition to the variation in the coefficient of friction. The second haptic signal can be sent to a second actuator or the same actuator(s) used to vary the coefficient of friction can generate the second haptic output.

CROSS-REFERENCE TO RELATED APPLICATIONS

This patent application claims priority to U.S. Provisional PatentApplication No. 61/159,482, entitled “Locating Features Using a FrictionDisplay,” filed Mar. 12, 2009, which is incorporated by reference hereinin its entirety.

This patent application claims priority to U.S. Provisional PatentApplication No. 61/262,041, entitled “System and Method for IncreasingHaptic Bandwidth in an Electronic Device” filed Nov. 17, 2009, which isincorporated by reference herein in its entirety.

This patent application claims priority to U.S. Provisional PatentApplication No. 61/262,038, entitled “Friction Rotary Device for HapticFeedback” filed Nov. 17, 2009, which is incorporated by reference hereinin its entirety.

This patent application is related to U.S. patent application Ser. No.______, filed the same day as the present application and entitled“Systems and Methods for a Texture Engine,” (Attorney Docket No. IMM354(51851-383720)), which is incorporated by reference herein in itsentirety.

This patent application is related to U.S. patent application Ser. No.______, filed the same day as the present application and entitled“Systems and Methods for Using Multiple Actuators to Realize Textures,”(Attorney Docket No. IMM355 (51851-383719)), which is incorporated byreference herein in its entirety.

This patent application is related to U.S. patent application Ser. No.______, filed the same day as the present application and entitled“Systems and Methods for Using Textures in Graphical User InterfaceWidgets,” (Attorney Docket No. IMM356 (51851-383718)), which isincorporated by reference herein in its entirety.

This patent application is related to U.S. patent application Ser. No.______, filed the same day as the present application and entitled“Systems and Methods for Providing Features in a Friction Display,”(Attorney Docket No. IMM357 (51851-383714)), which is incorporated byreference herein in its entirety.

This patent application is related to U.S. patent application Ser. No.______, filed the same day as the present application and entitled“Systems and Methods for Interfaces Featuring Surface-Based HapticEffects,” (Attorney Docket No. IMM359 (51851-383715)), which isincorporated by reference herein in its entirety.

BACKGROUND

Touch-enabled devices have become increasingly popular. For instance,mobile and other devices may be configured with touch-sensitive displaysso that a user can provide input by touching portions of thetouch-sensitive display. As another example, a touch-enabled surfaceseparate from a display may be used for input, such as a trackpad,mouse, or other device.

For example, a user may touch a portion of the display or surface thatis mapped to an on-screen graphical user interface, such as a button orcontrol. As another example, a gesture may be provided, such as asequence of one or more touches, drags across the surface, or otherrecognizable patterns sensed by the device. Although touch-enableddisplays and other touch-based interfaces have greatly enhanced devicefunctionality, drawbacks remain. For instance, even if a keyboard isdisplayed on a screen, a user accustomed to a physical keyboard may nothave the same experience while using the touch-enabled device.

SUMMARY

Embodiments include systems and methods for providing composite hapticeffects comprising a variation in a coefficient of friction of a touchsurface and one or more other outputs. By providing such surface-basedeffects, a device can provide a more compelling user experience than mayotherwise have been achieved.

In one embodiment, a method comprises detecting, using at least onesensor, a touch occurring in a touch area when an object contacts atouch surface. The method can further comprise selecting a compositehaptic effect to generate in response to the touch, the composite hapticeffect including at least one surface-based haptic effect and at leastone other effect. Based on the selected composite haptic effect, a firsthaptic signal can be sent to a first actuator to cause the firstactuator to vary a coefficient of friction of the touch surface. Themethod can further comprise, based on the selected composite hapticeffect, sending a second haptic signal to cause an actuator to provide asecond haptic output in addition to the variation in the coefficient offriction. The second haptic signal can be sent to a second actuator orthe same actuator(s) used to vary the coefficient of friction cangenerate the second haptic output. Another embodiment comprises atangible computer storage medium embodying program code executable by acomputing system for carrying out such a method.

These illustrative embodiments are mentioned not to limit or define thelimits of the present subject matter, but to provide examples to aidunderstanding thereof. Additional embodiments include systems anddevices configured to provide composite haptic effects and computerstorage media embodying program code for providing composite hapticeffects. Illustrative embodiments are discussed in the DetailedDescription, and further description is provided there. Advantagesoffered by various embodiments may be further understood by examiningthis specification and/or by practicing one or more embodiments of theclaimed subject matter.

BRIEF DESCRIPTION OF THE DRAWINGS

A full and enabling disclosure is set forth more particularly in theremainder of the specification. The specification makes reference to thefollowing appended figures.

FIG. 1A is a block diagram showing an illustrative system configured toprovide composite haptic effects.

FIGS. 1B and 1C show additional illustrative systems configured toprovide composite haptic effects.

FIG. 2A shows another illustrative system configured to providecomposite haptic effects.

FIG. 2B is a cross-sectional view of the system shown in FIG. 2A.

FIG. 3A shows another illustrative system configured to providecomposite haptic effects, while FIG. 3B is a cross sectional view of thesystem shown in FIG. 3A.

FIG. 4 is a flowchart showing illustrative steps in a method ofproviding composite haptic effects.

FIGS. 5A-5D show illustrative features that can be simulated usingsurface-based haptic effects.

FIGS. 6A-6H show illustrative features, specifically textures, that canbe simulated using surface-based haptic effects.

FIG. 7 is a flowchart showing illustrative steps in another method ofproviding composite haptic effects.

FIG. 8 a diagram illustrating an example of a reference file for use ingenerating composite haptic effects.

DETAILED DESCRIPTION

Reference will now be made in detail to various and alternativeillustrative embodiments and to the accompanying drawings. Each exampleis provided by way of explanation, and not as a limitation. It will beapparent to those skilled in the art that modifications and variationscan be made. For instance, features illustrated or described as part ofone embodiment may be used on another embodiment to yield a stillfurther embodiment. Thus, it is intended that this disclosure includemodifications and variations as come within the scope of the appendedclaims and their equivalents.

Illustrative Example of a Device Configured to Provide Composite HapticEffects

One illustrative embodiment of the present invention comprises acomputing system such as an iPod® portable music device or iPhone®mobile device, both available from Apple Inc. of Cupertino, Calif., or aZune® portable device, available from Microsoft Corporation of Redmond,Wash. The computing system can include and/or may be in communicationwith one or more sensors, such as an accelerometer, as well as sensors(e.g., optical, resistive, or capacitive) for determining a location ofa touch relative to a display area corresponding in this example to thescreen of the device.

As the user interacts with the device, one or more actuators are used toprovide tactile effects, including (but not limited to) composite hapticeffects. A composite haptic effect can comprise an effect that utilizesa variation in a coefficient of friction of a touch surface along withone or more other haptic outputs presented in combination with thevariation in coefficient of friction. The other output(s) may bepresented at the same time as the variation in the coefficient offriction, shortly before, and/or shortly after. The composite effect maybe perceived as a single effect or may be perceived as a plurality ofrelated effects.

For example, as a user moves a finger across the device, the coefficientof friction of the screen can be varied based on the position, velocity,and/or acceleration of the finger. Depending on how the friction isvaried, the user may perceive a feature and/or a texture. As aparticular example, the friction may be varied so that the userperceives a bump, border, or other obstacle corresponding to an edge ofan on-screen button. The composite haptic effect may further includetactile feedback as the on-screen button is pressed. As another example,a vibration-based effect may be generated so that a user perceives afeature such as a bump, border, or obstacle defining a control area,with a variation in the coefficient of friction used to indicate whenthe control is selected/changed. Still further, a combination ofvariation in friction and vibrotactile effects may be used incombination to create the perception of a single texture or otherfeature.

The variation in the coefficient of friction and other effect(s) may begenerated by the same actuator or by different actuators working inconcert. For instance, the coefficient of friction can be varied using apiezoelectric actuator, while vibrotactile effects are generated using alinear resonant actuator. As another example of examples of a hapticoutput that can be used alongside variations in the coefficient offriction, a portion of a touch surface can be raised/lowered (e.g.,using an actuator or shape memory alloy).

Illustrative Systems for Providing Surface-Based Haptic Effects inConjunction with Other Haptic Effects

FIG. 1A shows an illustrative system 100 for providing a compositehaptic effect. Particularly, in this example, system 100 comprises acomputing device 101 featuring a processor 102 interfaced with otherhardware via bus 106. A memory 104, which can comprise any suitabletangible (and non-transitory) computer-readable medium such as RAM, ROM,EEPROM, or the like, embodies program components that configureoperation of the computing device. In this example, computing device 101further includes one or more network interface devices 110, input/output(I/O) interface components 112, and additional storage 114.

Network device(s) 110 can represent any components that facilitate anetwork connection. Examples include, but are not limited to, wiredinterfaces such as Ethernet, USB, IEEE 1394, and/or wireless interfacessuch as IEEE 802.11, Bluetooth, or radio interfaces for accessingcellular telephone networks (e.g., transceiver/antenna for accessing aCDMA, GSM, UMTS, or other mobile communications network).

I/O components 112 may be used to facilitate connection to devices suchas a one or more displays, keyboards, mice, speakers, microphones,and/or other hardware used to input data or output data. Storage 114represents nonvolatile storage such as magnetic, optical, or otherstorage media included in device 101.

System 100 further includes a touch surface 116, which is in thisexample integrated into device 101. Touch surface 116 represents anysurface that is configured to sense tactile input of a user. One or moresensors 108 are configured to detect a touch in a touch area when anobject contacts a touch surface and provide appropriate data for use byprocessor 102. Any suitable number, type, or arrangement of sensors canbe used. For example, resistive and/or capacitive sensors may beembedded in touch surface 116 and used to determine the location of atouch and to determine other information, such as touch pressure. Asanother example, optical sensors may be used.

In this example, a first actuator 118 in communication with processor102 is coupled to touch surface 116. In some embodiments, first actuator118 is configured to output a haptic output varying a coefficient offriction of the touch surface in response to a first haptic signal.Additionally or alternatively, first actuator 118 may provide hapticoutput by moving the touch surface in a controlled manner. Some hapticoutputs may utilize an actuator coupled to a housing of the device, andsome haptic outputs may use multiple actuators in sequence and/or inconcert. For example, the coefficient of friction can be varied byvibrating the surface at different frequencies and/or amplitudes.Different combinations/sequences of variance can be used to simulate thefeeling of a texture, presence of another feature such as an obstacle orprotrusion, or to provide another effect.

Although a single first actuator 118 is shown here, embodiments may usemultiple actuators of the same or different type to vary the coefficientof friction of the touch surface. For example, a piezoelectric actuatoris used in some embodiments to displace some or all of touch surface 116vertically and/or horizontally at ultrasonic frequencies.

FIG. 1A also shows a second actuator 119. Second actuator 119 can beused to provide a second haptic output in addition to the variation inthe coefficient of friction as provided by the first actuator. Forexample, the second haptic output may comprise a vibrotactile hapticeffect such as a vibration of the device and/or touch surface 116. Asanother example, the second haptic output may comprise a change in thetouch surface 116, such as a change in temperature, color, or texturegenerated by raising/lowering portions of touch surface 116.

Actuators 118 and 119 may each comprise multiple actuators, and may beof the same or different types. Suitable actuator types include, but arenot limited to, piezoelectric actuators, shape memory alloys,electroactive polymers, flexible composite piezoelectric actuators(e.g., an actuator comprising a flexible material), electromagneticactuators, eccentric rotational mass actuators, linear resonantactuators, voice coil actuators, electrostatic actuators, and/ormagnetostrictive actuators. Although shown as separate elements 118 and119, a single actuator capable of varying the coefficient of friction oftouch surface 116 and generating another haptic effect could be usedinstead of separate first and second actuators. Additionally, inpractice one or more of the actuators could be embedded within touchsurface 116.

Turning to memory 104, exemplary program components 124, 126, and 128are depicted to illustrate how a device can be configured in someembodiments to provide composite haptic effects. In this example, adetection module 124 configures processor 102 to monitor touch surface116 via sensor(s) 108 to determine a position of a touch. For example,module 124 may sample sensor 108 in order to track the presence orabsence of a touch and, if a touch is present, to track the location,path, velocity, acceleration, pressure and/or other characteristics ofthe touch over time.

Haptic effect determination module 126 represents a program componentthat analyzes data regarding touch characteristics to select a compositehaptic effect to generate. For example, in some embodiments, an inputgesture comprising a sequence of one or more touches may be recognizedand correlated to one or more composite haptic effects. As anotherexample, some or all of the area of touch surface 116 may be mapped to agraphical user interface. Different composite haptic effects may beselected based on the location of a touch in order to simulate thepresence of a feature displayed on a screen by varying the friction oftouch surface and providing one or more other haptic outputs so that thefeature is “felt” when a portion of touch surface 116 mapped to a screenlocation containing the feature is encountered. However, haptic effectsmay be provided via touch surface 116 even if a corresponding element isnot displayed in the interface (e.g., a haptic effect may be provided ifa boundary in the interface is crossed, even if the boundary is notdisplayed).

Haptic effect generation module 128 represents programming that causesprocessor 102 to generate and transmit a haptic signal to actuator(s)118/119 to generate the selected composite haptic effect at least when atouch is occurring. For example, generation module 128 may access storedwaveforms or commands to send to actuator 118 and 119. As anotherexample, haptic effect generation module 128 may receive a desiredcoefficient of friction and utilize signal processing algorithms togenerate an appropriate signal to send to actuator(s) 118. Module 128may receive a desired haptic effect type and select an actuator 119 andcommand the selected actuator to provide the desired haptic effect.

As a further example, a desired texture may be indicated along withtarget coordinates for the texture. An appropriate waveform can be sentto a piezoelectric or other actuator to vary the coefficient of frictionusing high-frequency displacement, with another waveform sent to one ormore vibrotactile actuators to generate appropriate displacement of thetouch surface (and/or other device components) at a (relatively) lowerfrequency, or to provide a pop, ping, or other effect when the featureis encountered.

A touch surface may or may not overlay (or otherwise correspond to) adisplay, depending on the particular configuration of a computingsystem. In FIG. 1B, an external view of a computing system 100B isshown. Computing device 101 includes a touch-enabled display 116 thatcombines a touch surface and a display of the device. The touch surfacemay correspond to the display exterior or one or more layers of materialabove the actual display components.

In this example, a composite haptic effect is selected based on thecontent of a graphical user interface 130, which in this example shows a12-key keypad such as may be provided by a mobile device. Particularlyeach key may include a border 132 and interior 134. A composite hapticeffect can be selected to generate a variation in friction correspondingto interior 134 and a different haptic output can be used to indicatewhen a user has reached border 132. In this example, a feature 136 isincluded in the middle key (corresponding to the “5” key in a standardnumeric keypad). For instance, feature 136 may comprise a differenttexture from a texture of interior 134 or a simulated feature such as abump or gap serving as a “centering” feature. As a user moves a fingeracross the keypad, composite effects can be used to denote the borderbetween keys and to indicate when the “5” key is reached by theperception of feature 136.

As was noted above, a touch surface need not overlay a display. FIG. 1Cillustrates another example of a touch-enabled computing system 100C. Inthis example, a computing device 101 features a touch surface 116 whichis mapped to a graphical user interface provided in a display 122 thatis included in computing system 120 interfaced to device 101. Forexample, computing device 101 may comprise a mouse, trackpad, or otherdevice, while system 120 may comprise a desktop or laptop computer,set-top box (e.g., DVD player, DVR, cable television box), or anothercomputing system. As another example, touch surface 116 and display 122may be included in the same device, such as a touch-enabled trackpad ina laptop computer featuring display 122.

Whether integrated with a display or otherwise, the depiction of 2-Drectangular touch surfaces in the examples herein is not meant to belimiting. Other embodiments include curved or irregular touch-enabledsurfaces that are further configured to provide surface-based hapticeffects.

Returning to FIG. 1C, in this example a graphical user interface 138 isshown, comprising a menu bar 140 and buttons 142, 144, and 146. Forexample, the graphical user interface may comprise an on-screen menu fora home entertainment device, while touch surface 116 is included on aremote control or peripheral. As shown at 148, an area of the touchsurface is mapped to menu bar 140. When a user contacts area 148, acomposite haptic effect is provided to indicate that the menu bar isselected. For example, the coefficient of friction may differ as betweenarea 148 and areas 150, 152, and 154 (corresponding to buttons 142, 144,and 146, respectively), with vibrotactile or other feedback provided asa user moves through area 148 and crosses into one of areas 150, 152,and 154. As another example, vibrotactile or other haptic output can beused to generate different simulated textures in areas 148, 150, 152,and 154, with the coefficient of friction varied within each area asbuttons 142, 144, and/or 146 are actuated.

FIG. 2A shows another computing device 201 featuring a touch-enableddisplay 202, which again corresponds to a touch surface. FIG. 2B shows across-sectional view of device 201. As can be seen in FIG. 2B, in thisexample two actuators 218-1 and 282-2 are used to vary the coefficientof friction of touch surface 202 while a second actuator 219 providesanother haptic output via touch surface 202. For instance, actuator 219may provide a vibrotactile output such as a pop or ping through touchsurface 202 or may raise/lower one or more portions of the touch surface202. Additionally, FIG. 2B shows another exemplary actuator 222. Forexample, actuator 222 may be used in addition to or instead of one ormore of actuators 218-1, 218-2, or 219. For example, if actuator 219raises/lowers portions of display 202, actuator 222 may be used toprovide a low-frequency vibration output.

Returning to FIG. 2A, an example of providing a composite effect isillustrated at 220. Specifically, a finger 226 contacts the display andmoves to the position shown at 228, encountering a simulated feature230. As can be seen in FIG. 2B, as the finger moves, it initiallycontacts a first area 232 of the display, then a second area 234, and athird area 236. A composite haptic effect may be used to simulatefeature 230. For instance, while the finger encounters area 232 andmoves toward area 234, the coefficient of friction of display 202 may beincreased by adjusting output of actuators 218-1/218-2. As area 234 isapproached and then encountered, actuator 219 may be used to provide avibrotactile “pop” effect. As the finger moves across area 236, thecoefficient of friction of display 202 may be decreased.

A composite haptic effect may comprise a group of effects. For example,lines 238 and 240 indicate a path followed by the finger moving from 226to 228. In one embodiment, the composite haptic effect includes outputgenerated using actuator 219 and/or 222 if the finger moves outside thepath. For example, a variation in friction may be used to simulatefeature 230, with a vibration output provided while feature 230 issimulated if the user's finger crosses either of lines 238 and 240.

FIGS. 3A-3B depict an illustrative hardware architecture for a devicethat can provide composite haptic effects. In this example, the touchsurface comprises glass panel 302, although another transparent material(or non-transparent) material could be used. For instance, rather thanglass, a touchpad (i.e. touch-sensitive device) could be used. A pair ofpiezoelectric benders 318-1 and 318-2 are bonded to the glass. Use ofglass or another transparent material along with free space between thepiezoelectric benders can allow for use of a display (not shown) beneaththe glass. In some embodiments, the piezoelectric benders can becommanded to reduce the static coefficient of friction of glass 302 by42%. Some embodiments utilize a bipolar pulse width modulated signal at24 kHz, with varying amplitude used to vary the coefficient of friction.As an example, voltage can vary between −80 and +80 Volts at a frequencyabove 20 kHz, with friction variation from 0 to 60% depending on voltagemagnitude (or PWM magnitude to produce the voltage magnitude). Theseexample voltage, frequency, and variation ranges are for purposes ofexample only and are not intended to be limiting.

In the cross-sectional view of FIG. 3B, a plurality of second actuators319-1, 319-2, and 319-3 are illustrated. For instance, actuator 319-2may be bonded to panel 302 and can be used to provide low-frequencyvibration effects using an eccentric rotating mass (ERM) actuator and/orhaptic pops, jolts, and the like via a linear resonant actuator (LRA).Actuators 319-1 and 319-3 may be bonded to the case or housing of device301 and may comprise the same type of actuator as 319-2 or may bedifferent from actuator 319-2 (and/or one another). For example, in oneembodiment, actuator 319-1 comprises an ERM actuator, actuator 319-2comprises a shape memory alloy used to move panel 302, and actuator319-3 comprises an LRA actuator. Either or both actuators 319-1 and319-3 could be bonded to both panel 302 and the housing of device 301.

Illustrative Methods for Determining Haptic Effects to Provide

FIG. 4 is a flowchart showing an illustrative method 400 for providingan interface with composite haptic effects including at least onesurface-based haptic effects. Block 402 represents determining aposition of a touch in a touch area. For example, a processor mayutilize one or more sensors embedded in or viewing a touch-enableddisplay or surface to track a position of a touch on the surface. Basedon the current and/or past position of the touch, an interaction with agraphical user interface mapped to the touch area can be determined.

Based on the interaction, one or more composite haptic effects can beselected at block 404, with a composite effect achieved by varying thefriction of the touch surface along with providing one or more otherhaptic outputs.

For example, the touch may occur at a position in the touch surfacemapped to a particular texture or feature in a graphical user interface.The composite haptic effect may comprise a simulation of the texture orfeature. Additionally or alternatively, the composite haptic effect maycomprise an ambient effect (e.g., a texture/feature) and dynamic effect,such as output provided when a feature is initially encountered.

As an example, a gesture can be recognized as a sequence of one or moretouches or patterns of touch, such as based on a direction and length ofa swipe across the screen, a sequence of discrete touches in a pattern,or another recognizable interaction. If a gesture is recognized, acomposite haptic effect associated with the gesture can be selected. Forinstance, a “Z”-shaped touch trajectory may be recognized as a type ofinput gesture based on pattern recognition carried out by a processor ofthe device while the gesture is in progress. One or more compositehaptic effects may be associated with the “Z”-gesture in data accessibleto the processor indicating an effect to output while the gesture is inprogress and/or after the gesture is complete. For example, the data mayprovide for the surface to take on a texture or a change in friction asthe gesture nears completion. Additionally or alternatively, a textureor coefficient of friction of the display may change after the gestureis recognized in order to confirm input of the gesture.

At block 406, one or more haptic signals are accessed and/or generatedin order to provide a variation in the coefficient of friction and togenerate the second haptic output (or outputs). For example, a processormay access drive signals stored in memory and associated with particularhaptic effects. As another example, a signal may be generated byaccessing a stored algorithm and inputting parameters associated with aneffect. For example, an algorithm may output data for use in generatinga drive signal based on amplitude and frequency parameters. As anotherexample, a haptic signal may comprise data sent to an actuator to bedecoded by the actuator. For instance, the actuator may itself respondto commands specifying parameters such as amplitude and frequency.

As an example, for a simulated feature such as a gap, obstacle, ortexture the current pixel location and/or a projected pixel location forthe touch based on a velocity of the touch can be compared to a bitmapspecifying composite haptic effects for various pixel positions. Basedon the composite haptic effect(s), suitable haptic signals can beaccessed/generated to provide the output specified in the bitmap.

As another example, a current or projected location of a touch can becompared to data identifying the location of graphical user interface(GUI) features such as controls, textual content, boundaries, and thelike. Then, if a GUI feature is identified at the location, dataassociating one or more composite haptic effects to the feature can beaccessed. For instance, a processor may track the location of a touchand determine the touch is at or approaching a position in the toucharea mapped to a particular control (e.g., a button) in the graphicaluser interface. The processor can then consult a listing of interfaceelements to determine a composite haptic effect (e.g., a texture, afriction variation) associated with the button and, based on the hapticeffect, take further actions to generate the composite haptic effect.

Block 408 represents transmitting the haptic signal or signals to one ormore actuators to achieve the composite haptic effect by varying thecoefficient of friction and providing one or more other haptic outputs.For instance, if an analog drive signal is to be provided, a processorcan utilize an onboard D/A converter to create the signal. If a digitalcommand is provided to the actuator, an appropriate message can begenerated by an I/O bus of the processor. The haptic effect may be feltat the point of the touch and/or elsewhere. For example, if a two-fingerinput gesture is provided, the texture/coefficient of friction at thefirst finger may be changed in response to recognizing movement of thesecond finger.

In some embodiments, a baseline haptic signal may be sent to theactuator(s) to generate an ambient haptic effect even in the absence ofa selected haptic effect in order to enhance the range of potentialeffects the device can produce. Thus, transmitting a haptic signal maycomprise sending a “stop” command, a “zero” or minimal signal, oranother signal to the actuator to reduce intensity as appropriate.

As an example, use of certain actuators, such as piezoelectricactuators, may allow for reduction in the coefficient of friction of atouch surface but not an increase in the coefficient of friction. Toprovide a range of options, a baseline signal may be provided so thatthe “ordinary” friction level of the touch surface is below thecoefficient of friction the touch surface would have when static.Accordingly, haptic effects may be defined with respect to the baseline,rather than static, value. If maximum friction is desired, a “zero”signal may be sent to the piezoelectric actuator to stop movement of thesurface.

Surface-based haptic effects may take any suitable form. For example,some haptic effects may comprise variations in the friction of the touchsurface—some portions may be rendered “slicker” or “rougher” thanothers. As another example, vibrotactile effects may be used, such asvibrations or series of vibrations. Vibrotactile effects and/orvariations in friction may be used to simulate the feeling of distinctfeatures, such as boundaries or obstacles. For example, a boundary oredge may be simulated by an increase in friction, with the frictiondecreasing if the boundary is crossed (in some instances).

FIGS. 5A-5D each depict an illustrative simulated feature. FIG. 5A showsa simplified example in which the white area represents an area whereactuators will be activated, such as by using a non-zero voltage PWMsignal. For example, the white area may correspond to a virtual buttonin the middle of a touch pad, where a user's finger (or another objectin contact with a surface) will encounter a lower friction value. FIG.5B represents an inverses situation—the finger/object may navigatefreely in the white area, but may be slowed or stopped at the highfriction (black) area. This may, for example, allow a user to moreeasily locate a button or other location in the touch area.

FIG. 5C illustrates a simulated feature comprising a plurality ofgrooves. As a user's finger or another object moves horizontally acrossthe stripes, the finger/object will encounter increasing and decreasingfriction that is perceived as a series of grooves.

As was noted above, a computing system comprising a touch surfaceconfigured to provide surface-based haptic effects may determine effectsand signals in real time. For example, for any of FIGS. 5A-5D, thesystem may first determine if the touch position is inside the circleand, if so, provide a suitable output value (FIG. 5A) or cease output(FIG. 5B). Similarly, the system may provide the feature of FIG. 5C bydetermining if the touch occurs in an area with desired high-frictionand, if so, drive the actuator(s).

FIG. 5D presents a more complex pattern. For instance, the pattern inFIG. 5D may correspond to desired features associated with an array ofkeys, such as an array of mobile phone keys, a simulated keyboard, orother controls. Although real time rendering could be used for any ofFIGS. 5A-5D, more complex logic may be needed to render each specificcircle/button in the pattern. These and even more arbitrary patters mayincrease the complexity of programming and computation time. Thus, insome embodiments, the surface-based haptic effects can be determinedahead of time and stored in a file. At runtime, the file can be accessedbased on a touch position to allow for faster determination andgeneration of appropriate haptic signals. For FIG. 5D, such a file couldinclude data to drive the actuators to provide a first haptic effect(e.g., high friction) when the touch position is mapped to the circles,and the file could include data to drive the actuators to provide asecond effect (e.g., low friction) when the touch position is mapped toa location outside the circles

The features of FIGS. 5A-5D may be simulated using other types ofactuators in addition to or instead of piezoelectric actuators. Forexample, vibrotactile actuators may be used to provide a vibration, pop,pulse, click, or the like when the circles of FIG. 5A, 5B, or 5D areencountered and/or when the pattern of FIG. 5C is encountered (e.g., apop or click at each transition from black to white). The haptic effectsthat can be produced by the actuators can vary depending on the current,voltage, frequency as well as start and stop times. Such haptic effectsinclude, but are not limited to, vibrations, pulses, pops, clicks,damping characteristics, and varying textures. In an embodiment, themultiple actuators are utilized to generate different haptic effects fordifferent applications. For example, the two actuators are configured toprovide a vibration or pop upon the user's finger or stylus passing overthe boundaries of a graphical object (e.g. keyboard keys), as discussedabove.

The features of FIGS. 5A-5D may be simulated using composite hapticeffects. For example, a variation of friction may be used along with aclick, pop, or other response when an object touching the surface movesfrom black to white or vice-versa.

Vibrotactile effects and/or variations in friction may additionally oralternatively be used to simulate various textures. Additional detailregarding generation and use of textures can be found in U.S. patentapplication Ser. Nos. ______, and ______, referenced above and entitled“Systems and Methods for a Texture Engine,” (Attorney Docket IMM354(51851-383720)), “Systems and Methods for Using Multiple Actuators toRealize Textures,” (Attorney Docket No. IMM355 (51851-383719)), and“Systems and Methods for Using Textures in Graphical User InterfaceWidgets,” (Attorney Docket No. IMM356 (51851-383718)), respectively.

For instance, patterns of differing friction or patterns of vibrationmay be provided to mimic the feeling of textures such as brick, rocks,sand, grass, fur, various fabric types, water, molasses, and otherfluids, leather, wood, ice, lizard skin, metals, and other texturepatterns. Other textures not analogous to real-world textures may alsobe used, such as high-magnitude vibrotactile or other feedback when a“danger” texture is desired.

Generally, a texture can simulate a surface property by providing anoutput perceived as a spatially varying force. Thus, a texture iscontrasted to a vibration output intended to be directly perceived as atime varying force, though of course vibrations can be used insimulating texture.

As an example, a texture may simulate the force felt when a stick orfinger is moved over a grating. In one embodiment, the texture force canbe characterized by a programmer/developer using parameters that caninclude a magnitude and a grit. The magnitude specifies the amount offorce applied to the object encountering the touch surface (e.g., ateach “bump” of the grating). The grit is basically the spacing betweenforces (e.g., spacing between each of the grating bumps). Alternatively,additional command parameters can be provided to control the position ofthe “bumps” of the texture force. For example, information can beincluded to instruct the distance between bumps to vary exponentiallyover a distance, or vary according to a specified formula. Additionally,a given texture may be defined as a plurality of superimposed grits andmagnitudes.

FIG. 6A is an illustration of one of the textures that a texture enginemay generate according to one embodiment of the present invention. Theembodiment shown in FIG. 6A comprises brick. The texture of brick ischaracterized by having a rough irregular texture from bricks,punctuated with the feel of gritty valleys from the mortar. A system fora texture engine may generate the rough irregular texture of brick bydriving an actuator, such as a LRA, LPA, or FPA, with a random signalwith medium to high maximum variance while the user's finger is moving.In some embodiments, this variance may be adjusted for differentroughness. In some embodiments, the transition from brick to mortar maybe rendered by a high duration pop created by an ERM. Additionally, thecoefficient of friction may be varied between the brick and mortarportions. If the mortar is thick enough, a fine texture may be renderedby driving an actuator with a lower magnitude signal with a highervariance than that used to drive the actuator outputting the texture ofthe brick.

FIG. 6B is an illustration of one of the textures that a texture enginemay generate according to one embodiment of the present invention. Theembodiment shown in FIG. 6B comprises rocks. The texture of rocks ischaracterized by smooth surfaces punctuated with transitions as the usermoves from rock to rock. In order to output the texture of a rock, anactuator, such as an FPA and/or piezoelectric actuator can be used tocreate patches of low friction. Individual rocks may be rendered by anon-visual edge map of the displayed image, and outputting a highmagnitude haptic signal to an actuator, such as an LRA, LPA or ERM, whenthe touch-sensitive interface detects the user's movement. For example,a high-magnitude output can be provided whenever the touch-sensitiveinterface detects that the user's finger is transitioning from one rockto another. The coefficient of friction may be varied as between rocks,as well.

FIG. 6C is an illustration of one of the textures that a texture enginemay generate according to one embodiment of the present invention. Theembodiment shown in FIG. 6C comprises sand or sandpaper. Sand ischaracterized by a rough, gritty feel as well as the sensation a pile ofsand particles building up in front of the user's finger. In order tooutput the rough gritty texture, of sand, an actuator, such as an LRA,LPA or FPA is driven with a random signal with high maximum variancewhile the user's finger is moving. In some embodiments, the processormay adjust the variance of the signal to create different degrees ofroughness. To create the feeling of sand piling up, an actuator such asan FPA may be used. In such an embodiment, when user moves their fingeracross the touch screen, the processor will drive the actuator with asignal that starts with a low intensity and builds as the user movestheir finger in one direction. A piezoelectric or other actuator can beused to increase the coefficient of friction to simulate sand build-upas well.

In another embodiment, the texture shown in FIG. 6C may comprisesandpaper. Sandpaper is characterized by having a rough, gritty feel. Tocreate the rough, gritty feel the processor drives an actuator, such asan LRA, LPA or FPA with a random signal with high maximum variance. Insome embodiments, this signal is output only while the user's finger ismoving across the surface the touch sensitive interface. In someembodiments, the processor may adjust the variance of the signal tochange the level of roughness. Additionally, the roughness may begenerated or emphasized through abrupt changes in the coefficient offriction.

FIG. 6D is an illustration of one of the textures that a texture enginemay generate according to one embodiment of the present invention. Inthe embodiment shown in FIG. 6D, the texture comprises the texture ofgrass. Grass is characterized by a periodic light sensation that almosttickles the user's finger. In order to create the sensation of grass,the processor may drive an actuator, such as an FPA or piezoelectricactuator, with a signal configured to create patches of low frictionoverlaid with patches of grass. In some embodiments, the processor mayrender individual grass blades by having a non-visual edge map of thedisplayed image and outputting a low magnitude signal to an actuatorsuch as an LPA or ERM when the user interface detects movement betweenthe simulated blades.

FIG. 6E is an illustration of one of the textures that a texture enginemay generate according to one embodiment of the present invention. Inthe embodiment shown in FIG. 6E, the texture comprises the texture offabric. Fabric is characterized by a light smooth sensation. In order tocreate the sensation of the texture of fabric, the processor may drivean actuator such as an LPA or an LRA with low magnitude high frequencysignals as the user's finger moves across the surface of thetouch-sensitive interface. The “grain” of the fabric may be simulated byvarying friction levels of the display to provide lower friction whenmoving along the grain.

FIG. 6F is an illustration of one of the textures that a texture enginemay generate according to one embodiment of the present invention. Inthe embodiment shown in FIG. 6F, the texture comprises the texture ofwater or molasses. Water is characterized by having almost no sensation.However, water that is disturbed may splash around and hit against heuser's finger. To emulate the texture of water, the processor may drivean actuator such as an FPA to reduce the friction on the surface of thetouch-sensitive interface. To emulate the water sloshing, the processormay output the haptic signal only when the user is touching the screen.To emulate the texture of a more viscous fluid, such as molasses, oroil, the processor may drive the actuator with a signal configured toincrease the friction on the user's finger as it moves across thesurface of the touch-sensitive interface.

FIG. 6G is an illustration of one of the textures that a texture enginemay generate according to one embodiment of the present invention. Inthe embodiment shown in FIG. 6G, the texture comprises the texture ofleather. Leather is characterized by an overall smooth feeling thatcomprises the bumps and valleys of the surface of the leather. In orderto create the sensations of the texture of leather, the processor maydrive an actuator, such as an FPA or piezoelectric actuator, with asignal configured to output a haptic effect that reduces friction as theuser's finger moves across the surface of the touch-sensitive interface.The processor can output the cracks and bumps by driving the actuatorwith a very short low magnitude haptic signal at times when thetouch-sensitive interface detects that the user's finger is moving.

FIG. 6H is an illustration of one of the textures that a texture enginemay generate according to one embodiment of the present invention. Inthe embodiment shown in FIG. 6E, the texture comprises the texture ofwood. Wood may be characterized by an irregular bumpy texture punctuatedby a sharp transition as the user moves from board to board. In order tocreate the irregular bumpy texture, the processor may drive an actuatorsuch as an LRA, LPA, or FPA with a non-visual edge map of the displayedimage and drive the actuator with a very short low magnitude signal atvarious times when the user's finger is moving. To output the transitionfrom plank to plank, the processor may output a haptic signal configuredto cause the actuator to generate a high magnitude, short duration, pop.The friction of the touch surface can be varied when moving betweenplanks, but can be maintained (or only slightly varied) when movingalong the grain of the wood.

In other embodiments, haptic effects associated with different texturesmay be output. For example, in one embodiment, the processor maytransmit a haptic signal configured to cause the actuator to output ahaptic effect configured to cause the user to feel a texture associatedwith the texture of ice. Ice is characterized by low friction, in someembodiments, ice has a completely smooth texture, in other embodiments,ice comprises a fine low magnitude gritty texture. In order to createthe texture of ice, the processor may determine a haptic signalconfigured to cause the actuator to reduce the friction as much aspossible while the user moves their finger across the surface of thetouch-sensitive interface. In another embodiment, the processor maydrive an actuator such as an LPA or LRA, with a haptic signal configuredto output low magnitude effects while the user moves their finger. Theselow magnitude effects may be associated with imperfections or grit onthe surface of the ice.

In another embodiment, the processor may drive the actuator with asignal configured to cause the actuator to output a haptic effectapproximating the texture of lizard skin. Lizard skin is characterizedby an overall smooth sensation punctuated by transitions from bump tobump on the skin. In order to implement a haptic effect comprising thetexture of lizard skin, the processor may drive an actuator with ahaptic signal configured to cause the actuator to create patches of lowfriction on the touch-sensitive interface. The processor may rendercracks on the surface of the skin by outputting high magnitude hapticsignals periodically when the touch-sensitive interface detects that theuser's finger is moving across its surface. These high magnitude signalsmay approximate the cracks in the surface of the skin.

In yet another embodiment, the processor may drive the actuator with asignal configured to cause the actuator to output a haptic effectapproximating the texture of fur. Fur is characterized by a periodiclight sensation that is very soft to the touch. In order to implement ahaptic effect comprising the texture of fur, the processor may drive theactuator with a haptic signal configured to cause the actuator to outputa haptic effect configured to reduce the friction the user feels on thesurface of the touch-sensitive interface. The processor may furtherrender individual hairs outputting a low magnitude pulsing hapticsignals as the touch-sensitive interface detects the user's movement.

In yet another embodiment, the processor may drive the actuator with asignal configured to cause the actuator to output a haptic effectapproximating the texture of metal. Metal is characterized by a smoothlow friction surface that, in some embodiments, includes light grit. Inorder to implement a haptic effect comprising the texture of metal, theprocessor may drive the actuator with a signal configured to lower thefriction the user feels on the surface of the touch-sensitive interface.In some embodiments, the processor may render individual bumps byoutputting brief high magnitude haptic signals when the touch-sensitiveinterface detects that the user is moving over its surface. These briefhigh magnitude signals may approximate grit on the surface of the metal.

In yet another embodiment, the processor may drive the actuator with asignal configured to cause the actuator to output a haptic effectapproximating another sensation, for example, heat. In such anembodiment, the processor may output a haptic signal configured to causethe actuator to output a high frequency jolting effect when the usertouches elements of the display that are associated with heat.

FIG. 7 is a flowchart showing illustrative steps in another method 700of providing composite haptic effects. FIG. 7 is a flowchart showing anexemplary method 700 for providing a simulated feature by creating andusing a reference file. FIG. 8 shows an example of a reference filecomprising an array of pixels. Blocks 702 and 704 representpreprocessing—activities that occur prior to use of a reference file todetermine a composite haptic effect. In this example, a single referencefile is used to determine friction values and at least one other hapticoutput used in providing a composite haptic effect. Additionally, a“reference file” may comprise multiple files used together.

Block 702 represents creating a layout of a location and block 704represents storing the layout in an image file, such as an array ofpixels in a bitmap or other image file. For example, arbitrary shapesmay be “drawn” in order to specify desired friction values. In FIG. 8,white pixels are shown to indicate where no friction adjustment or otheroutput is intended, while shaded pixels indicate a value of a desiredcoefficient of friction or even a value usable to drive an actuator(e.g., a desired PWM voltage level, frequency, etc.). Alternatively,white pixels may indicate maximum drive, while various degrees ofshading indicate lower drive values, with black representing zero drive.In an embodiment, white pixels and black pixels only are used, with thecolors corresponding to on/off states of the actuators of a device.

In this example, different degrees of shading are represented bycross-hatching. In practice, each pixel may comprise multiple values(e.g., each pixel may have an RGB value), with the multiple valuesproviding different data, such as drive levels for different actuatorsand the like. Additionally, a reference file may include multiple layersfor specifying various parameters for each pixel position. For example,one layer may be used to define variations in the coefficient offriction, with one or more other layers used to define drive values orother information for use in outputting a vibrotactile or other hapticoutput. This example shows a relatively small number of pixels; inpractice, the array may comprise thousands or millions of pixels.

In this example, three buttons 802, 804, and 806 are shown, with buttonborders indicated by solid shading. Respective interiors 808, 810, and812 have different shading and are intended to represent differenttextures, friction values, or other effects corresponding to the buttoninteriors. A menu bar is shown with different shading at 814, along withtwo menu commands 816 and 818 with the same shading as one another,corresponding to different haptic effects that can be used to indicatewhen the menu bar is encountered and then when menu items areencountered. Although not shown here, transition areas between low andhigh (or high and low) friction or other effects could be included.

Returning to FIG. 7 and method 700, once a reference file is created, itcan be loaded into memory and read as shown at block 706 to determinecomposite haptic effects for output. For example, some or all of thepixel array may be maintained in working memory of a processor carryingout a position detection and feature simulation routine. In anembodiment, the pixel array is distributed alongside a correspondingimage of a graphical user interface. In additional embodiments, thepixel array is a layer or component of the graphical user interfaceimage, and in further embodiments the array is a separate file notassociated with a graphical user interface.

Block 708 represents determining a position of a touch. For example, asensor may provide data used to determine a pixel position of a touch inan array of pixels mapped to a touch area. Non-pixel coordinates may beused in identifying the location of a touch, with appropriate transformsused during the mapping step below.

Block 710 represents mapping the touch position to an entry (or entries)in the image file. For instance, the touch area may be mapped directlyso that a touch at pixel (x,y)=(10,12) results in accessing one or morepixel values in the image at image (x,y)=(10,12). However, more complexmappings may be used. For example, a touch position and velocity may beused to map a pixel value in the touch area to a different pixel valuein the image file. For instance, the size of the touch area and the sizeof the pixel array may differ, with a scaling factor used to map touchlocations to pixel values.

Block 712 represents activating one or more actuators to provide asurface-based haptic effect based at least in part on data from theimage file. For instance, the pixel value in the image file may bemapped to a desired coefficient of friction. A device carrying outmethod 700 may determine, based on the pixel position and the desiredcoefficient of friction, a suitable signal or signals to send to one ormore actuators to generate the desired coefficient of friction. Asanother example, the pixel value may indicate a drive signal moredirectly, such as a voltage/amplitude/frequency value or offset for aPWM signal to be sent to a piezoelectric actuator. Data of the array mayalso be configured for use in generating a drive signal for another typeof actuator.

As a more complex example, each pixel address may be associated withthree intensity values (i.e., RGB). Each of the three intensity valuescan be associated with a signal intensity/frequency for a correspondingactuator in some embodiments. As another example, some values mayspecify intensity and others specify duration of operation for the sameactuator. As a further example, different pixel intensity values may becorrelated to different desired textures or components used to driveactuators to simulate a single texture. For example, textures 808, 810,and 812 may be achieved using a combination of friction variations andvibrations. When a touch occurs at a location mapped to the areas, theappropriate actuator(s) can be used to generate the textures.

Method 700 may determine touch locations mapped to multiple pixels inthe image file. For example, a large touch may correspond to a range ofpixel addresses in the image file. Values from the range of pixeladdresses may be considered together, or analysis may be made to“pinpoint” the touch location and use values from a corresponding singlepixel address.

In this example, method 700 checks at 714 for one or more events tied toa haptic effect. As was noted above, a composite haptic effect mayutilize one or more actuators that provide a haptic output to confirm aselection, indicate a change in status, or otherwise provide feedback toa user. Block 716 represents using one or more actuators to providehaptic output corresponding to the event, and then continuing back to708 to determine the touch position.

For example, a user may move a finger or other object across a region ofa touch surface that is mapped to button 802 and on to a region mappedto button 804. Texture 808 may be generated and then, as thefinger/object moves to button 804, a click, pop, or other effect may beoutput to indicate the button borders. A click or button-press event maybe registered if the user lingers over a button for a predetermined timeand/or if the touch-enabled system registers a touch or increase inpressure. In response to the click event, a click, pop, or other effectcan be output to simulate the response of a physical button (or anotherresponse).

In some embodiments, a computing device featuring a touch surface withsurface-based haptic effects can output different surface-based hapticeffects based on sequences of inputs. Thus, the simulated features ofthe touch surface can vary based on a state of a device associated withthe surface. In some embodiments, this can be implemented using areference file with multiple layers; each layer can correspond to aparticular state. The states can be changed based on various inputconditions, for instance.

For example, a touch surface may be configured to act as a keypad, suchas on a mobile device. The keypad may feature three rows of keyscorresponding to numbers 1-9 and a fourth row with “0,” “*”, and “#”keys. For an initial state, the touch surface may be configured toprovide a centering feature, such as a higher friction level at the “5”key than in the remainder of the layout.

The computing device can be configured to change the state of the touchsurface in response to user input based on tracking the input relativeto the touch-sensitive area. For example, once the system determinesthat the user has found the “5” key, e.g. by detecting touching,hovering, or other activity indicating that the key has been located(but not necessarily selected), the surface-based effects can beprovided based on a different state. If a multi-layer reference file isused, for example, a different layer can be loaded into memory. In thesecond state, for instance, boundaries between keys can be provided sothat a user can proceed from the center to a desired key without theneed for visual feedback (although, of course, visual, auditory, orother feedback can be provided alongside any embodiments of the presentsubject matter).

The informational content or meaning of surface-based haptic effects canvary in various embodiments. For example, effects may be used toidentify particular portions of a touch surface mapped to areas in agraphical user interface, simulated keys or other controls, or may beprovided for aesthetic or entertainment purposes (e.g., as part of adesign and/or in a game). Effects may be provided for communicationpurposes as well. For example, Braille or other tactile-basedcommunications methods can be facilitated.

General Considerations

The use of “adapted to” or “configured to” herein is meant as open andinclusive language that does not foreclose devices adapted to orconfigured to perform additional tasks or steps. Additionally, the useof “based on” is meant to be open and inclusive, in that a process,step, calculation, or other action “based on” one or more recitedconditions or values may, in practice, be based on additional conditionsor values beyond those recited. Headings, lists, and numbering includedherein are for ease of explanation only and are not meant to belimiting.

Embodiments in accordance with aspects of the present subject matter canbe implemented in digital electronic circuitry, in computer hardware,firmware, software, or in combinations of the preceding. In oneembodiment, a computer may comprise a processor or processors. Theprocessor comprises or has access to a computer-readable medium, such asa random access memory (RAM) coupled to the processor. The processorexecutes computer-executable program instructions stored in memory, suchas executing one or more computer programs including a sensor samplingroutine, a haptic effect selection routine, and suitable programming toproduce signals to generate the selected haptic effects as noted above.

Such processors may comprise a microprocessor, a digital signalprocessor (DSP), an application-specific integrated circuit (ASIC),field programmable gate arrays (FPGAs), and state machines. Suchprocessors may further comprise programmable electronic devices such asPLCs, programmable interrupt controllers (PICs), programmable logicdevices (PLDs), programmable read-only memories (PROMs), electronicallyprogrammable read-only memories (EPROMs or EEPROMs), or other similardevices.

Such processors may comprise, or may be in communication with, media,for example tangible computer-readable media, that may storeinstructions that, when executed by the processor, can cause theprocessor to perform the steps described herein as carried out, orassisted, by a processor. Embodiments of computer-readable media maycomprise, but are not limited to, all electronic, optical, magnetic, orother storage devices capable of providing a processor, such as theprocessor in a web server, with computer-readable instructions. Otherexamples of media comprise, but are not limited to, a floppy disk,CD-ROM, magnetic disk, memory chip, ROM, RAM, ASIC, configuredprocessor, all optical media, all magnetic tape or other magnetic media,or any other medium from which a computer processor can read. Also,various other devices may include computer-readable media, such as arouter, private or public network, or other transmission device. Theprocessor, and the processing, described may be in one or morestructures, and may be dispersed through one or more structures. Theprocessor may comprise code for carrying out one or more of the methods(or parts of methods) described herein.

While the present subject matter has been described in detail withrespect to specific embodiments thereof, it will be appreciated thatthose skilled in the art, upon attaining an understanding of theforegoing may readily produce alterations to, variations of, andequivalents to such embodiments. Accordingly, it should be understoodthat the present disclosure has been presented for purposes of examplerather than limitation, and does not preclude inclusion of suchmodifications, variations and/or additions to the present subject matteras would be readily apparent to one of ordinary skill in the art.

1. A system comprising: a sensor configured to detect a touch in a toucharea when an object contacts a touch surface; a first actuator incommunication with the processor and coupled to the touch surface, thefirst actuator configured to provide a haptic output varying acoefficient of friction of the touch surface in response to a hapticsignal; a second actuator in communication with the processor andcoupled to at least one of the touch surface and a housing comprisingthe touch surface, the second actuator configured to provide a secondhaptic output, different from the output varying the coefficient offriction, in response to the haptic signal; and a processor incommunication with the first actuator, second actuator, and sensor, theprocessor configured to: select a composite haptic effect to generate,and transmit a haptic signal to at least one of the first and secondactuator to generate the selected composite haptic effect.
 2. The systemset forth in claim 1, wherein the first actuator comprises at least oneof a piezoelectric actuator, a shape memory alloy, an electroactivepolymer, a composite piezoelectric actuator, an electromagneticactuator, an eccentric rotational mass actuator, a linear resonantactuator, or a voice coil actuator; and wherein the second actuatorcomprises at least one of a piezoelectric actuator, a shape memoryalloy, an electroactive polymer, a composite piezoelectric actuator, anelectromagnetic actuator, an eccentric rotational mass actuator, alinear resonant actuator, or a voice coil actuator.
 3. The system setforth in claim 1, wherein the processor is further configured todetermine a position of the touch and wherein selecting the compositehaptic effect comprises determining a haptic output or combination ofoutputs that will simulate the presence of a feature in the touchsurface and at or near the determined position.
 4. The system set forthin claim 1, wherein the processor is configured to transmit a hapticsignal to both the first and second actuators to generate the selectedcomposite haptic effect.
 5. The system set forth in claim 4, wherein theprocessor is configured to send a first haptic signal to the firstactuator to vary the coefficient of friction to simulate a backgroundtexture while sending a second haptic signal to the second actuator tosimulate the presence of a feature overlaying the background texture. 6.The system set forth in claim 1, wherein selecting the composite hapticeffect comprises: determining a position of the touch in a display area;accessing a mapping file that maps each of a plurality of positions to arespective haptic effect; and identifying a composite haptic effectmapped to a selected position, the position selected based on thedetermined position of the touch.
 7. The system set forth in claim 6,wherein the position in the mapping file comprises a plurality ofvalues, including a first value used to generate a first haptic signalto send to the first actuator and a second value used to generate asignal for providing the second haptic output.
 8. The system set forthin claim 6, wherein the selected position is determined based on theposition of the touch and a velocity of the touch.
 9. The system setforth in claim 1, wherein the processor is further configured todetermine a pressure of the touch based on data from the sensor, whereinthe composite haptic effect is selected based at least in part on thepressure of the touch.
 10. The system set forth in claim 1, furthercomprising a display, the display comprising a plurality of pixels anddefining a display area, the display configured to output an image basedat least in part on a display signal, wherein the touch surfacecorresponds to the display or a material above the display, and whereinthe processor is configured to output the display signal.
 11. Acomputer-implemented method, comprising: detecting, using at least onesensor, a touch occurring in a touch area when an object contacts atouch surface; selecting a composite haptic effect to generate inresponse to the touch, the composite haptic effect including at leastone surface-based haptic effect; based on the selected composite hapticeffect, sending a first haptic signal to a first actuator to cause thefirst actuator to vary a coefficient of friction of the touch surface;and based on the selected composite haptic effect, sending a secondhaptic signal to cause an actuator to provide a second haptic differentfrom the variation in the coefficient of friction.
 12. The method setforth in claim 11, wherein the second haptic signal is sent to a secondactuator.
 13. The method set forth in claim 12, wherein the firstactuator comprises at least one of a piezoelectric actuator, a shapememory alloy, an electroactive polymer, a composite piezoelectricactuator, an electromagnetic actuator, an eccentric rotational massactuator, a linear resonant actuator, or a voice coil actuator; andwherein the second actuator comprises at least one of a piezoelectricactuator, a shape memory alloy, an electroactive polymer, a compositepiezoelectric actuator, an electromagnetic actuator, an eccentricrotational mass actuator, a linear resonant actuator, or a voice coilactuator.
 14. The method set forth in claim 11, further comprising:determining a position of the touch, wherein selecting the compositehaptic effect comprises determining a haptic effect that will simulatethe presence of a feature in the touch surface and at or near thedetermined position.
 15. The method set forth in claim 11, wherein thecomposite haptic effect comprises: varying the coefficient of frictionto simulate one of a background texture and a simulated feature; andusing the second haptic output to simulate the other of the backgroundtexture and the simulated feature.
 16. The method set forth in claim 11,wherein the first haptic signal is configured to cause the firstactuator to vary the coefficient of friction to simulate a feature andthe second haptic signal is configured to cause the second actuator toprovide tactile feedback in response to interacting with the simulatedfeature.
 17. The method set forth in claim 16, wherein the simulatedfeature comprises a simulated key and the second haptic signal isconfigured to provide tactile feedback in response to data indicatingthe simulated key has been pressed.
 18. The method set forth in claim11, wherein selecting the composite haptic effects comprises:determining a position of the touch; accessing a mapping file that mapseach of a plurality of positions to a respective plurality of hapticeffects; and identifying a plurality of haptic effects mapped to aselected position, the position selected based on the determinedposition of the touch.
 19. The method set forth in claim 11, furthercomprising: outputting a graphical user interface using a display,wherein the touch surface corresponds to at least one of: the display; amaterial above the display; or a peripheral device comprising an areamapped to at least a portion of the graphical user interface.
 20. Atangible computer storage medium embodying program code executable by acomputing system, the program code comprising: program code for causingthe computing system to track a location of a touch on a touch surface;program code for causing the computing system to select a compositehaptic effect to generate based on the interaction; program code forcausing the computing system to transmit a haptic signal to at least oneactuator to generate the selected composite haptic effect by varying acoefficient of friction of the touch surface; and program code forcausing the computing system to transmit a haptic signal to at least oneactuator to generate the selected composite haptic effect by generatinga second haptic output.